Center Based Clustering: A Foundational Perspective

In the first part of this chapter we present existing work in center based clustering methods. In particular, we focus on the k-means and the k-median clustering which are two of the most widely used clustering objectives. We describe popular heuristics for these methods and theoretical guarantees associated with them. We also describe how to design worst case approximately optimal algorithms for these problems. In the second part of the chapter we describe recent work on how to improve on these worst case algorithms even further by using insights from the nature of real world clustering problems and data sets. Finally, we also summarize theoretical work on clustering data generated from mixture models such as a mixture of Gaussians. 1 Approximation algorithms for k-means and k-median One of the most popular approaches to clustering is to define an objective function over the data points and find a partitioning which achieves the optimal solution, or an approximately optimal solution to the given objective function. Common objective functions include center based objective functions such as k-median and k-means where one selects k center points and the clustering is obtained by assigning each data point to its closest center point. In k-median clustering the objective is to find center points c1, c2, · · · ck, and a partitioning of the data so as to minimize Φ = ∑ x mini d(x, ci). This objective is historically very useful and well studied for facility location problems [13, 40]. Similarly the objective in k-means is to minimize Φ = ∑ xmini d(x, ci) . The k-means objective function is exactly the loglikelihood of data coming from a mixture of Gaussians. Hence, optimizing this objective is closely related to fitting the maximum likelihood mixture model for a given dataset. For a given set of centers, the optimal clustering for that set is obtained by assigning each data point to its closest center point. This is known as the Voronoi partitioning of the data. Unfortunately optimizing both these objectives turns out to be NP -hard. Hence a lot of the work in the theoretical community focuses on designing good approximation algorithms for these problems [13, 9, 26, 31, 40, 44, 45, 53, 17] with formal guarantees on worst case instances, as well as providing better guarantees for nicer, stable instances. In this chapter we discuss several stepping stone results in these directions, focusing our attention on the kmeans objective. A lot of the the ideas and techniques mentioned apply in a straightforward manner to the k-median objective as well. We will point out crucial differences between the two objectives as and when they appear. We will additionally discuss several practical implications of these results.